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ABSTRACT 

The technique of multivariate analysis is 
particularly suited to educational needs assessment research because 
it allows for the summarization of data across any number of learners 
or components of educational need to produce a single numerical index 
of need for each skill examined. In the needs assessment process, 
educational or training need is assumed to have three underlying 
dimensions: competence of the individual at a task or skill, 
relevance of the task or skill to the individual, ani 
individual's desire to further his or her learning oi 
skili>The use of multivariate analyris as a means f' 
educational need is superior to previous methods in 
earlier models could accommodate data for only two dimensions, in 
addition, the multivariate analysis method manifests anj increased 
sensitivity to changes in respondent distribution in the index of 
educational need. Finally, the model can accommodate different 
numbers and relative emphases of dimensions according to user-defined 
models and its method of computation is a relatively simple one. 
(This paper includes a discussion of the multivariate assessment 
technique as a model fdr assessing educational and training need, 
discussions of bivariate and multivariate cases of analysis, a 
computational example, a list of references, and six figures and one 
table illustrating various phases of the model and its standard error 
weicrhts.) (MN) 
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Abstract 



Instructional developers are becoming increasingly involved in needs 
assessment, determining what needs to be taught as well as determining hew best 
to teach and evaluate it. As increasingly complex models of the needs 
assessment process are proposed, methods must be developed to implement them* 
A three-component model of educational and training needs requires examining 
data on the competence of an individual at a skill, the relevance to the 
individual of the skill, and the individual's desire for further learning on 
the skill. Such data collections can be mind-boggling to analyze, especially 
if there are a significant number of skills and/or learners involved. This 
paper describes and illustrates a statistic which can summarize data across any 
number of learners or components of educational need, to produce a single 
numerical index of need for each skill. 
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As instructional developers increasingly become involved with the business 
of finding out what needs to be taught, as well as determining how best to ' 
teach and evaluate it, attention is focussed on the processes of needs 
identification and analysis. Whether in educational institutions or in 
training environments, instructional developers are being exposed to a growing 
body of literature reflecting increasing awareness of and attention to this 
not-yeit-all-that-clearly-understood technology (see, for example, Deden-Parker, 
1980; Gagne, 1977; Jones & Sommers, 1975; Kaufinan, 1972, 1977a, 1977b; Kaufinan 
& English, 1979; Kaufinan, Stakenas, Wager, & Mayer, 1981; Misanchuk, in press; 
Monette, 1977; Rossett, 1982; Roth, 1977; Scissons, 1982; Scriven & Roth, 1977; 
Spitzer, 1981; Witkin, 1976, 1977). 

The emergence of formalized models of needs assessment (e.g. , Kaufinan, 
1977a, 1977b; Kaufinan & English, 1979; Witkin, 1976, 1977) indicates the 
growing maturity of the field. As new models are explored, techniques for 
implementing them must be developed. For example, multidimensional 
conceptualizations of educational and training need have been put forward 
(Misanchuk, 1982; Misanchuk & Scissons, 1978; Scissons, 1982), but because of/ 
the difficulty in extracting comprehensible information from multidimensional 
data, have not been widely applied. 

We can illustrate the problem by focussing on the model wherein educational 
2 

or training need is assumed to have three underlying dimensions: competence 
of the individual at the task/skill 3 , relt. ce to the individual of the 
task/skill, and the individual's desire to further his or her learning of the 
task/skill (Misanchuk, 1982). High educational need is defined, in terms of 
these dimensions, when an individual demonstrates low competence, high 
relevance, and high desire. Similarly, a particular skill can be said to have 
high need associated with it when a pertinent group of individuals demonstrates 
— on the average— low competence, high relevance, and high desire. 

It is straight-forward enough, albeit not trivial, to collect information 
about each individual's competence at a task, the relevance of that task to tue 
individual, and the individual's desire for further training at the task. For 
example, five-point Likert-type scales could be used to collect data on each of 
the three dimensions. Once collected, however, the many data points— one for 
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each of the three dimensions, for each training task for which need is being 
assessed, for each individual in the learner group; a total of 3ts data points, 
where t * the number of skills whose needs are being assessed, and s ■ the 
number of potential learners involved— prove resistant to the analyst's 
traditional efforts. Several approaches to such analyses have been attempted 
and found wanting, including arbitrarily-defined cutting points with z-scores 
computed using the pooled inter-item variance (Misanchuk & Scissons, 1978), the 
odds ratio (Reynolds, 1977), and the dichotomized-additive coefficient 
procedure (LeSage, 1980; Misanchuk, 1980). If it were possible to reduce the 
3ts data points to, say, t data points, so that each skill would have 
associated with it an index of educational need which could be compared to the 
corresponding indexes for other skills, the results of the needs analysis would 
be much more comprehensible. 

A Multivariate Model of Educational Need 

For simplicity in language, the discussion here will focus on a three- 
dimensional model of educational need. However, the argument can be extended 
to n dimensions. 

' Consider the cube in Figure 1, respresenting the postulated three- 
dimensional configuration (Misanchuk, 1982) of an educational need. According 
to the model, the stippled cell in the forward-most, upper-right corner 
represents the position of highest need. In other words, if all individuals 
being surveyed responded to each of the three dimensions being measured in such 
a way as to place themselves in that forward, upper-right cell, the skill about 
which the individuals were being asked would have associated with it an 
extremely high educational need. 



Insert Figure 1 about here. 



Conversely, if all individuals were in the rear-most, lower left cell 
(which is completely hidden from view in Figure 1) , the skill under 
consideration would be said to have virtually no educational need associated 
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with it. 

t. 

Problems of interpretation occur when, as is usual in real life, the data 
distribute themselves across all three dimensions. Some examples will 
illustrate these difficulties, but first a diversion is necessary to establish 
a convention which will make it easier to describe individual cells in an array 
such as Figure 1. 

For convenience in communication, the individual cells in the three- 
dimensional matrix will be designated in terms of their row, coluim, and layer 
(or stratum) numbers. Thus, (1,5,1) represents the cell in row 1, column 5, 
and layer 1 (the forward-most). Cell (1,5,1), is of course, the cell 
indicating highest need. The cell associated with lowest need, mentioned 
earlier, is (5,1,5), and the cross-hatched cell in Figure 1 is (2,4,1), 'diile 
* the cell with horizontal shading is (4,5,2). Cell (3,3,3) is in the exact 

center of the cube. 

New the examples: It should be intuitively obvious that a skill for which 
all responses fall into cell (3,3,3) has a lower educational need than does one 
for which all responses fall into cell (4,2,4) , but hew does it compare to a 
skill for which all responses fall into cell (3,2,2)? What if, for one skiil, 
- there are 17 people in (3,2,2) , while for another skill, there are 15 in cell 
(3,3,3): which skill demonstrates the higher need? In real life, we can 
expect to find varying numbers of individuals in each of the 125 cells in 
Figure 1, further complicating matters. 

One solution to the problem wherein only two dimensions were incorporated 
into the model of educational need (Misanchuk, in press) adapted a 
proportionate reduction in error statistic, del (7) (Hildebrand, Laing, and 
Rosenthal, 1977a, 1977b) , for use in needs analysis. The ensuing statistic, V N 
(the proportionate reduction in error index of educational need) , can collapse - 
information from two dimensions into a single statistic, which represents the 
educational need associated with the skill under consideration. For example, 
educational need can be defined in terms of both relevance and competence, and 
a single value can be used to represent the average measure on both dimensions 
simultaneously. Unfortunately, the symmetry of the mathematics involved in the 
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analysis permits only a two-dimensional model of need. 

The remainder of this paper describes an extension of V^, which can be 
applied to n-dimensional situations. For the sake of simplicity and clarity, 
the development of the procedure will use a three-dimensional model, but the 
extension is straightforward. 

The underlying logic of the PRE approach involves predicting the 
probability that certain combinations of a joint distribution will occur, then 
testing to see how closely the prediction matches observations. In the current 
application, we postulate a high educational need for a given skill, then 
compare the observed distribution of responses given by learners to the 
prediction. In more concrete terms, we begin by predicting that all 
respondents will answer in cell (1,5,1) , then apply a mathematical procedure 
based on a proportionate reduction in error approach (Hildebrand, et al., 
1977a, 1977b) to determine how accurate our prediction was. The result of the 
procedure, the statistic V N , can be used as an index of the educational need 
associated with the skill under consideration, provided certain assumptions are 
made and their implications incorporated into the calculation (Misanchuk, in 
press) . 



The Bivariate Case 

In the two-component case, using ccnpetence and relevance for the sake of 
illustrating the procedure, high need would be defined as concomitant low 
competence and high relevance. Our prediction of high need therefore 
translates into a prediction of low competence and high relevance. In graphic 
terms, we would be dealing with only the front-most layer of the cube in Figure 
1 in the two-caiponent case; high need is indicated by cell (1,5,1). Any 
learners 1 responses falling in other cells would constitute errors in our 
prediction. Obviously, the further the cells in question are from (1,5,1) in 
both directions, the more severe the effects of the errors on the accuracy of 
the prediction. 

The PRE procedure permits the assignment of varying "degrees of error" to 
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each cell to accomodate the Increased severity of effect. Thus cell (1,5,1) 
would have no error associated with the prediction (i.e., an error weight of 
0), while cells (1,4,1) and (2,5,1) could be given error weights of 0.177, cell 
(2,4,1) could be given an error weight of 0.250, and so forth. The "worst" 
error, cell (5,1,1), could be considered a "whole" error, and have an error 
weight of 1.0 . While the numerical values of the error weights are 
arbitrary— others couli be assigned at the discretion of the researcher— they 
are not chosen capriciously: arguments have been presented for the values 
suggested here (Misanchuk, in press). 

The statistic del is defined by Hildebrand et al. (1977a, 1977b) as 

Expected errors - Observed errors 



Expected errors 

with both expected and observed errors taking into account the cell error 

weights mentioned above. The expected error rate is determined by the marginal 

4 

totals of the matrix in the same way as in chi square. 
In formal terms, del is defined as 
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where is the error weight for cell (i,^) (V^j = l for every "whole" error 
cell; 0 < < l fbr every "partial" error cell) , £jj is the probability of a 
randomly sampled observation falling into cell (i,j), and P. and p . are the 
expected marginal probabilities for the rows and columis, respectively. The 
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proportionate reduction in error index of educational need, V N (Misanchuk, in 
press) , is computationally equivalent to Formula (1) , but assumes certain known 
values for the error weights and the expected marginal probabilities. 

For the bivariate case, it matters not which of the two variables is 
designated the dependent variable and which is designated the independent; both 
ways, the computed value of V is the same (Hildebrand, et al., 1977b, p. 71). 
(Indeed, in the needs analysis situation, the designation of variables as 
dependent and independent is quite meaningless.) Unfortunately, this symmetry 
does not hold for the multivariate case, and despite the lack of meaning 
associated with the .terms dependent and independent fo*. needs analysis, some 
accommodation must be made for that fact. 



Hie Multivariate Case 

Hildebrand, et al. (1977a, 1977b) show that the PRE approach can be used 
with n-dimensional arrays by collapsing them into two-dimsnsxonal ones: the 
independent variables sure reconfigured as a single (new) independent variable 
composed of all the possible combinations of the original independent 
variables. 

To illustrate, assume a dependent variable Y with levels y^, ^ ^3' ^ 
two independent variables X and Z with levels x^ and x^, and z^, z^, and 
respectively. Graphically, this example arrangement would be that displayed in 
Figure 2(a). The strategy for collapsing is simple: just change the labels on 
the columns to reflect the conjunction of the two variables. Instead of having 
two columns labelled — one under x^ and one under Xj — we label the two 
coluims x^ & z^t and xj & Z\» 



Insert Figure 2 about here. 



Thus, the. collapsing process advocated by Hildebrand et al. (1977a, 1977b) 
creates a new variable V whose elements are 3^ & z^, 3^ & Zj, & Z3, ^ & Si» 
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x 2 & ^ £2 & -3' as s* 10 ^ ^ Figure 2(b) . The computation of V then 
proceeds in the same manner as in the bivariate case, using Y as the dependent 
variable (with three levels) and V as ttye independent variable (with six 
levels) . 

Formally, Hildebrand et al. (1977b, p. 261) define the trivariate V as 



R C S 

^ X X w p 

pi j=i j^i -iUi -lis 

V ^ 1 R C S , ^ 

V f' V 

i»i i-i k=i ~- • 



Formula (2) applies when Y is the dependent variable (with i. levels) and X 
and Z are the independent variables (with j. and k levels, respectively) . If 
either X or z is chosen as the dependent variable, seme of the dot notation in 
the formula must change to reflect the change in dependent variable. 
Specifically, the dot notation in the denominator of Formula (2) is P . P, 
when X is the dependent variable, and P k when Z is the dependent 
variable » 6 The different formulas thus generated yield different values for V, 
depending on which of the three variables is designated as the dependent 
variable, hence the symmetry problem alluded to earlier. 

By analogy to the bivariate case, the approach to the computation of the 
multivariate PRE index of educational need considers that cell (1,5,1) is 
errorless (i.e., its error weight W ■ 0), and that all other cells have 
associated error weights which increase as one moves away from cell (1,5,1) in 
any one or some combination of the n dimensions. For the trivariate case, cell 
(5,1,5) is assumed to have an error weight of 1.0, and all others have 
proportionate weights. The weights in Figure 3 are based on a simple gecnetric 
proportioning through three dimensions of the distance between cells (1,5,1) 
and (5,1,5), and are presented here as the proposed standard distribution of 
error weights for trivariate needs analysis using five-point scales. 
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Obviously, other distributions of weights could bo used If there seems a "need 
tct-do so. 



Insert Figure 3 -ibout here. 
- w ^ - 

To handle the symmetry problem, we propose that, since-the notion of 
dependent and independent variables is in any case irrelevant to the needs 
analysis situation, the n variables involved in the analysis (typically three: 
relevance, competence, and desire) each be treated in turn as the dependent 
variable, and the resulting values of V be averaged. 

Coaputational ExaqaLe 

TO illustrate the procedure, an intuitive approach will be taken to a 

step-by-step calculation of a three-dimensional generic , which will ' 

subsequently be contrasted with the calculation of a three-dimensional PRE 

index of educational need, 7 M . This will be followed by the expression of a 

-3 

computational formula for the rv-dimensional PRE index of educational need, V N , 

-n 

and by several examples designed to provide some understanding of the range of* 
values of V N that can be expected for different distributions of data. 

Consider the array of data displayed in Figure 4 (a) . The dependent 
variable, competence, is displayed as the row variable, and the columns are 
formed by all possible combinations of levels of relevance and desire. The 
marginal totals are conputed by simply sunning across the rows and columns. 
Entries of ,Q have been eliminated both within cells and in marginals to avoid 
visual clutter. 



Insert Figure 4 about here. 



By substitution into Formula (2)/ working by coluims and substituting only 
for non-empty cells (since, for empty cells, Zrapf *" °)» ^ using error weights 
from Figure 3, we have *J the numerator of the second term 

(.1443) 3/63 + (.2041) 9/63 + (.4564) 2/63 + (.0000) 7/63 + (.1.443) 11/63 + 

(.4787) 1/63 + (.2041) 2/63 + (.2500) 6/63 + (.1443) 3/63 + (.2041) 1/63'+ (3) 

(.3227) 4/63 + (.4330) 2/63 + (.3227) 4/63 + (.6770) 3/63 + (.6292) 1/63 + 

(.4564) 1/63 + (.8416) 2/63 + (.6124) 1/63 . 



J 



In the denominator of the second term, the row marginals, P. , and the 

column marginals, P , are multiplied by the appropriate cell error weight 

"""•Jul 

-AilS for every cel1 where both P^ and P^ are non-zero: 



.1443) (15/63) (14/63) 




(.2041) (36/63) (14/63) 


+ 


(.3227) (7/63) 


(14/63) 


+ 


.4564) (5/63) (14/63) 


+ 


(.0000) (15/63) (18/63) 


+ 


(.1443) (36/63) 


(18/63) 


+ 


.2887) (7/63) (18/63) 


+ 


(.4330) (5/63) ( 


1^/63) 


+ 


(.4564) (15/63) 


(1/63) 


+ 


.4787) (36/63) 


(V63) 


+ 


(.5401) (7/63)' 


fl/63) 


+ 


(.6292) (5/63) 


(V63) 


+ 


.2041) (15/63) 


(8/63) 


+ 


(.2500) (36/63) 


(8/63) 


+ 


(.3536) (7/63) 


(8/63) 


+ 


.4787) (5/63) 


(8/63) 


+ 


(.1443) (15/63) 


(8/63) 


+ 


(.2041) (36/63) 


(8/63) 


+ 


.3227) (7/63) 


(8/63) 


+ 


(.4564) (5/63) 


(8/63) 


+ 


(.4082) (15/63) 


(2/63) 


+ 


.4330) (36/63) 


(2/63) 


+ 


(.5000) (7/63) 


(2/63) 


+ 


(.5951) (5/63) 


(2/63) 


+ 


.2887) (15/63) 


(4/63) 


+ 


(.3227) (36/63) 


(4/63) 


+ 


(.4082) "(V63) 


(4/63) 


+ 


.5204) (5/63) 


(4/63) 


+ 


(.6124) (15/63) 


(3/63) 


+ 


(.6292) (36/63) 


(3/63) 


+ 


.6770) (7/63) 


(3/63) 


+ 


(.7500) (5/63) 


(3/63) 


+ 


(.4564) (15/63) 


(1/63) 


+ 


.4787) (36/63) 


(V63) 


+ 


(.5401) (7/63) 


(V63) 


+ 


(.6292) (5/63) 


(1/63) 


+ 


.4330) (15/63) 


(V63) 


+ 


(.4564) (36/63) 


(1/63) 


+ 


(.5204) (7/63) 


(V63) 


+ 


.6214) (5/63) 


(V63) 


+ 


(.7217) (15/63) 


(2/63) 




(.7360) (36/63) 


(2/63) 


+ 


.7773) (7/63) 


(2/63) 


+ 


(.8416) (5/63) 


(2/63) 


+ 


(.5951) (15/63) 


(1/63) 


+ 


.6124) (36/63) 


(1/63) 


+ 


(.6614) (7/63) 


(V63) 


+ 


(.7360) (5/63) 


(V63) 


• 



(4) 



Simplifying , 



0.2643 

V - 1 ■ ■ ■ ■ = 0,0476 with competence as the dependent variable. 

0.2775 

Recasting the same data so that relevance is the dependent (row) variable 
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and competence and desire are the independent (column) variables, we get the 
matrix found in Figure 4 (b) . Because the error weights migrate with the 
cells*— they are properties of the cells, determined by the relative values of 
the variables that define the cells, rather than properties of the order in 
which the cells are arranged for display— the error weights substituted into 
Formula (2) are now those in Figure 5(a) . The numerator of the second term 
becomes 

(.1443) 3/63 + (.0000) 7/63 + (.204.1) 9/63 + (.1443) 11/63 + (.4564) 2/63 + 
(.2041) 2/63 + (.1443) 3/63 + (.4787) 1/63 + (.2500) 6/63 + (.2041) 1/63 + 
(.3227) 4/63 + (.*330) 2/63 + (.3227) 4/63 + (.4564) 1/63 + (.6770) 3/63 + (5) 
(.6292) 1/63 + (.6124) 1/63 + (.8416) 2/63 

= 0.2643, 

and the denominator of that term becomes 



.4330) .(fc/63) (10/63) 


+ 


(.2887) (2/63) (10/63) 


+ 


(.1443) 


(24/63) (10/63) 


+ 


.0000) (31/63) (10/63) 


+ 


(.4564) (6/63) (20/63) 


+ 


(.3227) 


(2/63) (20/63) 


+ 


.2041) (24/63) (20/63) 


+ 


(.1443) (31/63) (20/63) 


+ 


(.6124) 


(6/63) 


(2/63) 


+ 


.5204) (2/63) 


(2/63) 




(.4564) (24/63) 


(2/63) 


+ " 


(.4330) 


(31/63) 


(2/63) 


+ 


.4564) (6/63) 


'(5/63) 


+ 


(.3227) (2/63) 


(5/63) 


+ 


(.2041) 


(24/63) 


(5/63) 


+ 


.1443) (31/63) 


(5/63) 


+ 


(.4787) (6/63) 


(8/63) 


+ 


(.3536) 


(2/63) 


(8/63) 


+ 


.2500) (24/63) 


(8/63) 


+ 


(.2041) (31/63) 


(8/63) 


+ 


(.5401) 


(6/63) 


(4/63) 


+ 


.4330) (2/63) 


(4/63) 


+ 


(.3536) (24/63) 


(4/63) 


+ 


(.3227) 


(31/63) 


(4/63) 


+ 


.5401) (6/63) 


(6/63) 


+ 


(.4330) (2/63) 


(6/63) 


+ 


(.3536) 


(24/63) 


(6/63) 


+ 


.3227) (31/63) 


(6/63) 


+ 


(.6292) (6/63) 


(V63) 


+ 


(.5401) 


(2/63) 


(V63) 


+ 


.4787) (24/63) 


(V63) 


+ 


(.4564) (31/63) 


(1/6 J) 


+ 


(.6770) 


(6/63) 


(3/63) 


+ 


.5951) (2/63) 


(3/63) 


+ 


(.5401) (24/63) 


(3/63) 


+ 


(.5204) 


(31/63) 


(3/63) 


+ 


.7500) (6/63) 


(1/63) 


+ 


(.6770) (2/63) 


U/63) 


+ 


(.6292) 


(24/63) 


(1/63) 


+ 


.6124) (31/63) 


(1/63) 


+ 


(.7360) (6/63) 


U/63) 


+ 


(.6614) 


(2/63) 


(V63) 


+ 


.6124) (24/63) 


(1/63) 


+ 


(.59.51) (31/63) 


(V63) 


.+ 


(.8416) 


(6/63) 


(2/63) 


+ 


.7773) (2/63) 


(2/63) 


+ 


(.7360) (24/63) 


(2/63) 


+ 


(.7217) 


(3V63) 


(2/63) 





» 0.2777 . 
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Notice that, as one might expect from perusal of tormula (2) , the numerator 
has the same terms as it did when competence was the dependent variable, 
although they appear in a different order due to the reconfiguration of the 
data. 



Insert Figure 5 about here. 



With relevance as the dependent variable, then, 
0.2643 

V » 1 » 0.0483 . 

0.2777 

In a similar manner, the data can be reconfigured again, as in Figure 4(c) , 
making desire the (dependent) row variable, and relevance and competence the 
(independent) column variables. Substituting these reconfigured values into 
Fornula (2) , and using\the error weights as now arranged in Figure 5 (b) , yields 
a value of V » 0.0557 . 

So far in this example, the > marginal totals have been substituted directly 
into the computational formula j. However, as in the bivariate case, it makes 
less sense to use the observed \marginals than it does to make some assumptions 
about the marginals (see Misanchuk, in press) . 

By analogy to the bivariate case, both' competence and relevance will be 
assumed to mono ton ically increase along both dimensions as we move away from 
cell (1,1,1). The rationale for this assumption is that the task analysis 
underlying the needs assessment process should have ensured that the tasks used 
as a basis for the needs identification procedure were, by and large, quite 
relevant to the job roles of the individuals being assessed, and that the 
process of natural selection that obtains in hiring and firing tends to place 
more-ot-less competent individuals into job roles. Therefore expected 
probability distributions of 0, 0.1, 0.2, 0.3, and 0.4 are probably reasonable 
(Misanchuk, in press) . 
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With respect to desire, however, there is no justification to assume a 
monotonically increasing distribution. It does seem reasonable to assume a 
normal distribution, on the grounds that in any population, one could expect to 
find varying degrees of enthusiasm ior undertaking further training on any 
particular skill, regardless of relevance of the skill or competence in it. 7 
Superimposing the unit normal curve on a five-point scale gives the expected 
marginal distribution for desire of 0.0359, 0.2384, 0.4514, 0.2384, and 0.0359, 
which is proposed here as the standard expected marginal distribution for 
desire to undertake further training, and which will be used in the remainder 
of this paper. Again, if the needs analyst has information about the 
population that mkes this assumption unwarranted, that information could be 
translated into a more acceptable expected marginal distribution for desire. 

With these assumptions about marginal distributions, substitution into 
Formula (2) is changed somewhat, since instead of the observed marginals, we 
must use the assumed marginals. For Figure 4(a) , the row marginals are 
therefore 0, 0.1, 0.2, 0.3, 0.4 . The coluim marginals are formed by the 
multiplication of the expected marginals for the two variables making up each 
of the columns, namely, 0, 0.1, 0.2, 0.3, 0.4 for relevance, and 0.0359, 
0.2384, 0.4514, 0.2384, 0.0359 for desire. Thus 0, 0.0036, 0.0072, 0.0108, 
0.0144, 0, 0.0238, 0.0476, 0.0714, 0.0952, 0, 0.0451, 0.0902, 0.1353, 0,1804, 
0, 0.0238, 0.0476, 0.0714, 0.0952, 0, 0.0036, 0.0072, 0.0108, and 0.0144 form 
the column marginals for Figure 4 (a) . 

With these marginals and the error weights in Figure 3, the denominator of 
Formula (2) for the data in Figure 4(a) becomes 



(.5774) 


(0) 


(0) 


+ 


(.5951) 


(.1) 


(0) 


+ 


(.6455) 


(.2) 


(0) 


+ 


(.7217) 
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+ 
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(.4) 
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(.2) 
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+ 


(.7217) 


• 
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(.3) 


(.0144) 
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(.0165) 


(.4) 
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• 
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Note that the numerator of Formula (2) is unaffected by the assumption of 
expected marginal distributions, and remains 0.2643 . Substituting Term (7) 
into Formula (2) and simplifying, we get 



N 



1 - 



0.2643 



0.5748 



0.5402 . 



Similar substitutions of expected marginals— arranged in appropriate order 
for the data displayed in Figures 4 (b) and 4 (c) —and error weights from Figures 
5(a) and 5(b), respectively, give values of V N of 0.5402 and 0.5414 . 



The mean of the th-:ee values of V M is considered the trivariate 

N 

proportionate reduction in error index of educational need, V N , and is equal 



in this case to 0.5406 . 



-3 



In more formal terms, the trivariate PRE index of educational need, V M , is 

-3 



V 1 



V V V 
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£4 In £4 W £ 
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2, i 2, w. . p . 

i-L i-l k-L iiJi ili 

R C S 

f f f 

i-i i-i k-L -LL- 



(8) 



given some pre-specified expected marginal distributions and error weights. 

Notice that Formula (8) is composed of the three different versions— in 
terras of dotted subscripts— of the V N version of Formula (2) , which are then 
averaged. 

The extension to n dimensions is reasonably easy, if notationally complex. 
To avoid the unnecessary complication of extending the notation into a general 
formula for the n-dimensional case, we will simply describe the extension 
conceptually: The fraction 1/3 becomes 1/n, where n ■ the number of 
dimensions; there should be n terms inside the square brackets, each with the 
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sane numerator shown in Formula (8), except that the subscripts for both W and 
P should number n (i.e., the subscripts should be and for four 
dimensions, Hnign gjikmn for five d ^»nsions, etcT) , and there should be n 
summation signs. Also, this denominators of the n terms within the brackets 
should have the dot notation 

-i . . . -.jkm' ^.i. . Si.taB' K . k . £ii.m' . .m' ^ijk . ' 
respectively, for the four-dimensional case; 

it 

-i . . . . -. jkmn ' K^. . . -i . tan' -. .k. . -ij..mn' 2. . .m. £jjk .n' £. . . .n -ijkm . 

for the five-dimensiona?. case; and so forth. Again, there should be n 
summation signs and n subscripts for W. 

Some Illustrative Examples of V„ 

To provide a feeling for the kinds of values one can expect for V M , the 

-3 

data in Figure 6, along with the error weights in Figure 3. and the expected 

marginal distributions discussed earlier, have been substituted into Formula 

(8) . The data in the various matrixes in Figure 6 -have been arbitrarily 

arranged in a way that keeps the total number of observations constant, but 

systematically moves increasing numbers of responses away from the cell 

associated with highest need. Table 1 shows, as might be expected, that the 

values of 7 M decrease as we move from the data arrangement in Figure 6 (a) to 
-3 

that in Figure 6(f) . 



Insert Figure 6 about here. 



Insert Table 1 about here. 
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Conclusion 

The proportionate reduction in error (PRE) index of educational need, V v . 

(Misanchuk, in press) , which could accomodate data from only two dimensions, 

has been defined in a more general way, permitting n-dimensional data to be 

analyzed. The expanded definition—the multivariate proportionate reduction in 

error index of educational need, V N —-permits the analysis of data generated by 

-n 

a three-component nodel of educational and training need (Misanchuk, 1982; 
Misanchuk & Scissons, 1978; Scissons, 1982), or, more generally, allows for the 
analysis of data generated by a model which incorporates four or more 
dimensions. 

A standardized set of error weights (Figure 3) was proposed for the 

three-component case, and the expected marginal distributions which must be 

specified by the needs analyst (monotonically increasing for relevance of the 

task or skill to be taught and for the individual's competence at the task or 

skill, and normally distributed for desire to undertake further education or 

training in the task or skill) were identified. The statistic allows the 

researcher to deviate from the recommended values if there seems good reason to 

do. so. For example, if it weLe part of the model of educational need that 

desire should count only half as heavily toward determining educational need as 

the other need components, the set of error weights can be adjusted to 

accomodate the reduced influence of desire. Or, if the researcher had 

evidence to shew that, say, the expected marginal distribution for desire 

should be something other than a normal distribution, the adjustment could be 

made: nothing in the mathematics of determining V N is affected by the 

— n 

specification of alternative error weights or expected marginal distribution 
schemes. 

The increased sensitivity of V N to changes in respondent distribution as 

-n /-s 

cenpared to competing methods (Misanchuk, 1980) , itjs ability to acccmncdate 
different numbers and relative enphases of dimensions according to user-defined 
models, and its relative sinplicity of computation 8 argue for its application 
in instructional development projects, especially where large numbers of 
learners and/or large numbers of sk ills must be studied. 
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Footnotes 

1. The author would like to acknowledge the assistance of R. A. Yackulic with 
some of the technical details of the paper. T. M. Schwen graciously read 
and cemented upon an earlier draft of this paper. 

2. We acknowledge the distinction between training and education, but for the 
sake of ease in reading we will, in this paper,- henceforth avoid specifying 
each of them by treating the two as synonymous. From the point of view of 
needs assessment, whether the need is for training or for education is 
largely irrelevant, as the needs assessment procedures are virtually the 
same. The language used in this paper typically refers to the assessment 
of job-related training. 

3. The terms task and skill will be used in this paper to mean approximately 
the same thing: a job-related activity that can be learned. Tasks or 
skills— as the terms are used here— include everything from specific 
psychomotor activities to complex groups of activities that may involve 
cognitive and/or psychomotor (and perhaps even affective) components. For 
example, while typing at 60 wgn would certainly qualify as a skill under 
the definition used here, so would preparing an income tax return , or 
counselling employees . Equally, the terms are meant to apply to such 
multi-faceted activities as using computers , and wage and salary 
administration . Hereafter , the terms skill and task will be used 
interchangeably. . 

4. This statement is true for ..the .generic del as- defined by Hildebrand et al. 
(1977a, 1977b). However, for the needs analysis case, the expected error 
rate is specified by the needs analyst as monotonically increasing along 
both dimensions as we move away from the upper left hand corner of the two- 
dimensional matrix (Misanchuk, 1983). The point will be raised again later 
in this paper. 

5. The dot notation indicates summation over all values of the subscript which 
is replaced by the dot. For example, P. means the proportion of i 

considered over all values of P . means the proportion of j over all i. 

""•J. 

6. Multiple dot notation is read in a manner similar to single dot notation. 
Hence, P 4 is the proportion of i over all categories of 1 and k, P . is 

the proportiu i of i over all i and Jc, and so on. 

7. Some experience with the problem suggests that there may be a more complex 
relationship between conpetence and desire for further training than this 
assunption acknowledges i It often seems that learners want to learn more 
about something that they already do reasonably well (the "preaching to the 
converted 1 * syndroms) , making the assumption of a normal distribution 
somewhat questionable. However, the assumption will suffice until further 
research can establish nore accurately the exact relationship between 
learners' competence and their desire for further training. At such tine, 
an accommodation can be made by simply changing the expected marginal 
distribution. 
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8* The computation of V M is well within the sec** of a hand calculator, 

-3 

albeit somewhat tedious when numerous skills are being studied. Existing 
computer programs can be acquired, or can easily be written to ease the 
labor. 
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Table 1 

Values of Trivariate Del 
for Example Data in Figure f 



Matrix V M 

h 

(a) 0.9100 

(b) 0.8621 

(c) 0.8143 

(d) 0.6743 

(e) 0.3655 

(f) -0il673 
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Figure 1. Representation of a three-dimensional model of educational need. 
Each dimension of the cube is arbitrarily divided into five categories 
for convenience in data-collection; soma other number of categories could 
be used. 
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Figure 2. Two ways of graphically representing three-dimensional data. 
Once represented in the arrangement in^fb)', the data can be used to 
ccnpute a two-dimensional V in the conventional way. 
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Figure 3. Proposed standard error weights for trivariabe proportionate reduction in error index 
of educational need using five-point scales. Entries are X 1CT 4 . .decimals points omitted for 
clarity. 
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Figure 4. 


Example data for the three-dimensional case. 
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Figur e 5. Reconfigured error weights. 
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Figure 6. Exanple data illustrating systematically decreasing educational need. 
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